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ABSTRACT 

This paper presents a model for the identification 
and validation of competencies, designed as an all-inclusive 
framework, which may be adapted for specific situations. The criteria 
for including data in the validation process are as follows: (a) if 
the data are to be employed in making decisions about individuals or 
groups, all available evidence should be gathered before attempting 
to interpret the results; (b) if the data are to be used for 
prediction or selection, the instruments should be validated in the 
specific situation for which they are to be employed; and (c) ideas 
about what is being measured as well as what the instruments measure 
must change as new evidence is forthcoming. LOGOS (Language for 
Optimizing Graphically Ordered Systems) , used in the development of 
this model, outlines the processes for meeting the proposed project 
objectives. The LOGOS model, consisting of two cells— developing 
competency statements and establishing construct validity and 
reliability, is outlined at the conclusion of this paper. A 21-item 
bibliography is included. (PD) 
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The existence of performance based teacher and administrator certification 
CO programs needs little documentation Legislatures, state departments of educa- 
tlon and other regulatory agencies are mandating that candidates for professional 

• r-H education credentials demonstrate specific competence In predetermined teaching 

or administrative activities. 

• ^ An Interesting assumption is made by those supporting the competency move- 
. ^ ment. While no claims of perfection for the competency approach are made, the 

^ i I dissatisfaction with earlier teacher and administrator program formats Is readily 
acknowledged. The essence of the argument Is that the competency approach takes 
a giant step toward better preparation through Its Insistence on actual perfor- 
mance. Such performance resting on a foundation of knowledge and skills Is seen 
dS a necessary and logical extension of previous efforts. 

A nagging question remains, however. At present there Is no solid evidence 
to support the contention that those In possession of the predetermined compe- 
tencies are better teachers or administrators than those unable to demonstrate 
the competencies. This question must remain unanswered for the time being 
because prior questions of validity and reliability must first be settled. The 
best that can be said Is that logic alone supports the ontentlon that those who 
can demonstrate some teaching or administrative competencies are better teachers 
or administrators than those who cannot. 

Research In competency based curriculums for administration and supervision 
Includes studies to Identify and specify competencies, develop models for Iden- 
tifying competencies, develop models for competency based training programs, 
develop actual training programs, and evaluate methods of attaining competencies. 
Of the works reviewed, eleven dealt with the Identification and statement of 
administrative competencies. Five Identified competencies through a search of 
the literature, one used the critical Incident technique, and one used a jury of 
experts. Others did not specify the source of their competency statements. 
Validation procedures consisted almost exclusively of judgments by a jury or 
panel of experts with little attention to the generation of statistical evidence 
of val Idlty. 

A notable exception was the study by Cook and Van Otten, University of Utah, 
(1972) "...to Identify some of the prime competencies of the secondary school 
U-v principal ship and to report the attitudes of school district superintendents, 
^As^ secondary schopi principals and secondary school teachers concerning their Impor- 
<^ tance" (p. xl). Correlation data were used to determine the perceptions of the 
\ subjects as to what "are" and what "ought" to be prlmve competencies of secondary 
school principals and to rank the competencies In the order of Importance. 

'Cv The Validation. Mode I 

Precise statements of competencies and the development of assessment Instru- 
r>< ments to measure them can do much to further the professional status of education, 
^) General acceptance of these statements, however, will depend largely on the 
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evidence that possession of the competencies actually makes a difference In the 
quality of performance. A more rigorous and systematic approach to the valida- 
tion process Is one way to provide this evidence. 

A joint committee of the American Educational Research Association, the 
American Psychological Association, and the National Council of Measurements 
Used In Education Identified four types of validity (Technical Recommendations..., 
1954). The first, content validity. Is the representativeness or sampling ade- 
quacy of the universe of content. The second and third, predictive and concurrent 
validity, are criterion related, characterized by prediction to an outside cri- 
terion and by checking a measuring Instrument either now or In the future by some 
outcome. The fourth type, construct validity. Is Involved In Interpreting a 
measure of some attribute or quality which Is not operationally defined. 

The consistency with which an Instrument measures a construct does not satisfy 
but limits validity. An Instrument must be reliable In order to be valid, but 
reliability does not ensure validity. Chase and Ludlow (1966) Identified three 
types of reliability coefficients: (I) Internal consistency, (2) equivalence, 
and (3) stability. Determination of acceptable reliability coefficients is a 
major part of the val Idatlon process. 

The validation model illustrated in Figure I has been designed as an alf- 
Incluslve framework which may be limited or adapted for specific situations. 
It Is not necessary In every case to utilize all the processes in every cell to 
achieve an acceptable measure of validity. Tyler (1963) suggested three criteria 
for Including data In the validation process: 

1. If the data are to be employed in making decisions about individuals or 
groups, all available evidence should be gathered before attempting to 
Interpret the results. 

2. If the data are to be used for prediction or selection, the Instruments 
should be validated in the specific situation for which it is to be 
emp loyed . 

3. Ideas about what Is being measured as well as what the instruments 
measure must change as new evidence Is forthcoming. 

o 

The systems methodology LOGOS was used In the development of the model for 
the Identification and Validation of Competencies. LOGOS is an acronym derived 
from the title Language for Optimizing Graphically Ordered Systems (Silvern, 
June, 1969). The model Is composed of two major cells: 1.0 — Develop Com- 
petency Statements, and 2.0 — Establish Construct Validity and Reliability 
(see Figure I). The processes for meeting the proposed project objectives are 
outlined and ordered in the LOGOS model. 

1.0 - Develop Ctompetency Statements. Researchers work In the field with 
administrators to develop competency statements. 

I.I - Assess Competency Needs. 

I.ll - Conduct Job App./sJs of Function. Job Analysis by function 
seems appropriate since the use of fund Ions as a basis for classification pro- 
mises more discrete statements of competencies. 
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1.12 - Identify Critical Incidents. Critical Incidents provide a 
reality-oriented base for the analysis of competency needs. 

1.2 - Develop Competency Categories. Competency categories are devel- 
oped through analysis of the job functions and critical Incidents developed In 
cell i.l. 



1.3 - Specify Competencies for Categories and Identify Behaviors. Three 
processes will be combined to Identify specific competencies and Illustrative 
behaviors. The processes are: 

1.31 - Conduct Literature Review. 

1.32 - Analyze Critical Incidents. 

1.33 - Review Job Analyses. 

By using a combination of the three techniques the Investigators will be able to 
include the "Ideal" as well as the"real." 



1.4 - Develop Competency Rating Scale. This rating scale Is used to 
gather data for determining the construct validity and reliability of the com- 
petencies. 

1.41 - Establish Content Validity. Content validity Is accomplished 
using the following techniques: 



1.41 I - Conduct Panel of Expert Rating. 

1.412 - Conduct Rating by Practicing Administrators. 

1.413 - Develop List of Additional Competencies. 



1.5 - Develop Revised Competency Rating Scale. Establish criteria for 
adding and removing competencies from the rating scale. 

1.51 - Add Competencies. Competencies mentioned often by the judges 
should be added to the rating scale. 

1.52 - Eliminate Competencies. Competencies ranked low by a majority 
of the experts are removed from the competency rating scale. 

2.0 - Establish Construct Validity and Reliability. Stanley and Hopkins 
(1972) have delineated four steps in the process of developing a measure of 
psychological constructs and establishing Its validity. The steps Included: 
(I) develop a set of tasks or Items based on a rational analysis of the con- 
struct, (2) derive testable predictions regarding the relationship between the 
construct and other variables, (3) construct empirical studies of these theo- 
retical predictions, and (4) eliminate tasks or Items that operate contrary to 
the theory (or revise the theory) and proceed again with steps two and three. 
The first step in the Stanley and Hopkins process Is accomplished In cell 1.0 
of the LOGOS model. The remaining three steps are completed In cell 2.0. 

2.1 - Derive Testable Hypotheses. Samples of possible hypotheses are: 
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1. Principals of Innovative schools will demonstrate a greater number of 
leadership competencies than principals of traditional schools. 

2. Building level administrators ranked In the top one-third of the dis- 
trie; by superordlnates will be rated higher on the competency rating scale 
than a random sample of building level administrators. 

3. There will be a positive correlation between ratings on the competency 
scale dnd other validated Instruments developed to measure leadership. 

2.2 - Gather Raw Data for Empirical Testing. Data gathering Is done In 
the one major ->tep Included In cell 2.21. 

2.21 - Administer Instruments. Peers, subordinates and super- 
ordlnates, trained observers, and the subjects themselves rate selected groups 
of administrators on the competency rating scale and other selected leadership 
measurement In^>truments. Data from the ratings are analyzed as follows: 

2.3 - Analyze Data. 

2.31 - Test Hypotheses. 

2.32 ' Determine Discrimination Index. 

2.33 - Conduct Factor Analysis. Nunnally (1972), Cronbach and 
Meehl (1955), and Kerllnger and Kaya (1959) identified factor analysis as an 
Important part of construct validation. This Is an effective tool for elim- 
inating "gaps and overlaps" (Mclntyre, undated, p. 18) In the competency state- 
ments and for determining the reality of the factors. 

2.34 - Conduct AnalysJs of Variance. Analysis of variance tech- 
niques are used to determine that portion of the variance which can be attributed 
to the construct being measured. 

2.35 - Correlate with Other Validated Instruments. Hypotheses 
concerning the correlation of the competencies with other validated Instruments 
which measure related constructs are generated and tested. 

2.4 - Conduct Reliability Testing. A measurement Instrument must be 
reliable In order to be valid, but reliability does not ensure validity. 
Reliability coefficients are established In this cell. 

2.41 - Determine Coefficient of Internal Consistency. 

2.42 - Determine Coefficient of Stability. 

2.43 - Determine Inter-rater Reliability. 

If data analyses lead to major revisions In the competency rating scale, 
the process feedback to cell 1.4 and cells 1.5, 2.1, 2.2, 2.3 and 2.4 are 
repeated. 
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